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ABSTRACT 

Image analysis often starts with some 
preliminary segmentation which provides a 
representation of the scene needed for further 
interpretation. Segmentation can be performed in 
several ways, which are categorized as pixel- 
based, edge-based, and region-based. Each of 
these approaches are affected differently by 
various factors, and the final result may be 
improved by integrating several or all of these 
methods, thus taking advantage of their 
complementary nature. 

In this paper, we propose an approach that 
integrates pixel-based and edge-based results by 
utilizing an iterative relaxation technique. This 
approach has been implemented on a massively 
parallel computer and tested on some remotely 
sensed imagery from the Landsat-Thematic 
Mapper (TM) sensor. 

1. INTRODUCTION 

After pre-processing of some original data, 
image segmentation is the process which 
generates a spatial description of an image as a 


set of specific parts, regions or objects. The 
"segmented" output is then utilized by a higher- 
level image interpretation process. There is no 
single standard approach to segmentation which 
would be "successful" for any type of data, but 
some general methods have been developed 
based on the two main characteristics of regions 
or objects in an image: 

(1) each region or object exhibits an internal 
uniformity with respect to some image 
property (e.g., gray level, color, texture), 

(2) each region or object presents some 
contrast with its surroundings. 

These two properties lead to three different 
types of segmentations, pixel-based, region- 
based and edge-based segmentations. 

Each of these approaches is affected 
differently by various factors. Pixel-based 
methods form their decision only based on the 
information given at each pixel, while the two 
other types of segmentation take into account the 
information contained in the surrounding pixels. 
Usually in a pixel-based approach, all of the 
original information is utilized, thus avoiding a 
selection process. Such methods are also easier 
to integrate in a learning process, but their main 
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drawback is that they do not take into account 
spatial information. Conversely, edge- and 
region-based approaches base their decision on 
spatial information. Edge-based methods 
measure the variation of intensity between pixels 
belonging to different objects; they may produce 
excellent results for unevenly illuminated images, 
but they also can be very sensitive to noise. 
Region-based approaches, which measure the 
internal uniformity of some intensity or texture 
function, often produce spurious segmentation 
under non-uniform lighting conditions, but are 
usually less sensitive to noise. 

In general, these three types of 
segmentation may be improved by integrating 
them and by taking advantage of their 
complementary nature. We previously proposed 
an approach that integrates region segmentation 
and edge detection results by interpreting a binary 
tree representation (Le Moigne, 1992), thus 
producing a refined region segmentation. This 
algorithm has been tested on Landsat-TM data. 
The integration of edge and region data may also 
be performed by a relaxation method and has 
been proposed in (Le Moigne, 1989); in the work 
described in this paper, we refine this relaxation 
method for the purpose of integrating edge and 
classification information and we implement it on 
a massively parallel computer, the MasPar MP-1. 
Then, we test this approach on remotely sensed 
imagery, such as Landsat-TM data. 

2. RELAXATION TECHNIQUES 
a. Overview 

A large number of iterative relaxation 
schemes have been proposed to improve the 


results given by such basic processes as edge 
detection, region segmentation or pixel 
classification (Davis, 1981; Hummel, 1987; 
Faugeras, 1981; Peleg, 1980; Zucker, 1977). 
The principle of these algorithms is to utilize 
contextual information for iteratively changing 
the initial labeling of the objects in a scene toward 
optimal labeling. We will concentrate on 
relaxation methods for which the decisions at 
each point are taken in a probabilistic fashion. 
This general class of relaxation techniques is 
described in (Davis, 1981). Let us assume that 
we have a set of N objects {O j , O 2, . . . , On) 
(e.g., the pixels) to be labeled into one of L 
classes {Ci, Ci, . . . , Cl) and that p"[Ckl is the 
probability that the object Oj is assigned to the 
class Ck at the iteration n. The principle of the 
relaxation algorithms, then, is to build a series of 
probability sets {p"[Ckl ;1 s is N; 1 < k<L), 
where each new iteration step, n, adjusts the 
probabilities according to the contextual 
information, and these probabilities satisfy the 
conditions (1) and (2): 


and 


for every i, for every k, 
0<p"[C k ]<l, (1) 

L 

for every i, ^p"[Clc] = 1. (2) 

k=1 


Thus, the problem is to define the updating 
formula for p"[Ckl. The simplest updating (which 
we use in this work) is described below. 

The relaxation scheme assumes that the 
class assignments of each object depend on the 
class assignment of the "other" objects; the 
"other" objects can be defined, for example, by 
neighboring objects. Therefore, we define 
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c[i,k;j,l] as the compatibility coefficient between 
the object Oj with the label Ck and the object Oj 
with the label Q. We assume that the c[i,k;j,l]’s 
belong to the range interval [0,+l] and are 
positive if there is compatibility and equal to 0 if 
there is high incompatibility. The coefficient 
c[i,k;j,l] can be defined as a conditional 
probability, p(i e Ck/j e Cj), which provides a 
probabilistic framework. Let qj’tCkl be the global 
compatibility for the object Oi with the label Ck; it 
is defined by 

! j=V[i] 1=L n 

q"[Ck]=^j.Z ^ c[i,k;j,l] Pj [Q] (3) 

where V[i] is the number of objects in the 
neighborhood of object Oj. Then q"[Ckl is the 
"increment" which is applied to update p" [Ckl 
and compute the new probability set {p- n+1 ^fCkl} 
(see (Davis, 1981) for details): 

p"[C k ]xq"[C k ] 

pTVu-h; (4 > 

p"[Cl] X q" [C1] 

1=1 

This multiplication still ensures that the 
conditions (1) and (2) are satisfied. Besides, if 
the global compatibility of the object Oj with the 
label Ck is higher than all the compatibilities of 
Oi with the other labels Ci, then the p"[Ckl's 
increase relative to the other p"[Ci]'s. That means 
that this scheme provides an overall improvement 
of the labeling but it does not guarantee the 
convergence toward stable labeling. 


Other relaxation schemes (Faugeras, 1981; 
Peleg, 1980) utilize different formulas or 
different frameworks. For example in (Faugeras, 
1981), another way of updating the probabilities 
p?[Ckl is given; the principle of this other method 
is to minimize a criterion by the "projected 
gradient" optimization method. The criterion 
measures the ambiguity and the consistency of 
the current labeling at each step. This algorithm 
provides us with a converging sequence of 
probabilities P n . Starting from an initial point P°, 
the method converges toward a local minimum in 
the vicinity of P°. 

b. Utilizing Relaxation to Integrate 
Disparate Information 

We now describe how a relaxation 
technique, such as the one described above, can 
be utilized to integrate knowledge from edge 
detection and pixel classification. 

Previously, Zucker and Hummel (Zucker, 
1977) simultaneously used edge and region data 
in a relaxation process for labeling dots. Their 
goal was to provide a low-level description of the 
roles of dots in cluster analysis. Unlike our 
approach, they used these two types of data 
"equally", i.e., both edges and regions defined 
the labels and the initial probabilities. There were 
ten different labels, with eight edge labels at 
various orientations, one "region" label called 
"interior point" label, and one "noise" label. 

In our approach, the labels are all region 
labels, and edge data are utilized to update the 
labeling by way of the compatibility coefficients. 
This approach tends to be more general and this 
algorithm can be very easily applied to the fusion 
of various types of data. In our work, the 
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definition of the labels and the initial probabilities 
are provided by a neural network pixel 
classification (Chettri, 1992), then the relaxation 
process updates the initial labeling by using some 
edge detection results, e.g., from a Canny 
operator (Canny, 1986). These standard 
techniques, pixel classification and edge 
detection, can easily be changed without altering 
the definition of the overall relaxation algorithm. 

Once the initial probabilities are given, the 
fusion is realized by computing the 
compatibilities between neighboring pixels. 
Coefficients c[i,k;j,l] in formula (3) represent the 
compatibilities between object Oj with the label 
Ck and object Oj with the label Cj. If a relaxation 
is performed only on regions, these coefficients 
represent the compatibilities of neighboring 
pixels belonging to given regions. If both regions 
and edges are considered, both information can 
be integrated into the compatibility coefficients by 
utilizing the following equation: 

c[i,k;j,l]= p r (Ck/Ci) x Fk&l(i,j) (5) 

where p r (Ck/Ci) is the "region-probability" 
(independent of edges) of i belonging to the class 
Ck if the neighbor j belongs to the class C|; this 
probability could be estimated from some 
previous ground truth data. The last term in Eq. 
(5), Fk&i(ij), is a function that varies in [0,1] and 
is: 

- closer to 1 if Ck = Q (or, more generally, if 
regions Ck and Ci are similar in tone) and i is not 
an edge point, or if regions Ck and C| are 
different and i is an edge point, 

- closer to 0 if Ck = Q and i is an edge point, or 
if regions Ck and C\ are different and i is not an 
edge point. 


For example, F can be defined by: 

Fk&l(i j) = Kk&l x Mag(i) 

+ (1- K k &l ) x (1- Mag(i) ) 

where Kk&l is equal to 0 if k=l and equal to 1 if 
k*l, and Mag(i) is the magnitude of the gradient 
computed at the point i and normalized between 0 
and 1. 

The magnitude of the gradient at neighboring 
point j would also be taken into account and we 
could utilize the following formula: 

Fk&lO j)= Kk&l x Mag(i) x Mag(j) 

+ (1- K k &l ) x (1- Mag(i) x Mag(j) ). 

This updating has not been implemented yet, but 
will be considered in future work. 

Therefore, both region and edge 
information participate simultaneously in the 
updating of the labeling. Results are presented in 
Section 4. 

3.PARALLEL IMPLEMENTATION 

Computation time is the main concern of 
relaxation techniques. However these techniques 
are characterized by parallel local processing 
which can be easily implemented on an 
architecture that favors computations between 
adjacent pixels (Fishier, 1987). The method 
discussed in this paper has been implemented on 
a MasPar MP-1. The MasPar Parallel Processor 
is a fine-grained, massively parallel SIMD 
architecture, with 16,384 parallel processing 
elements arranged in a 128x128 matrix and 
connected by an eight nearest neighbors 
interconnection network. 
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The parallel implementation of the 
relaxation algorithm is straightforward, with the 
quantities p"[Ckl and q"[Ckl computed in parallel 
at each pixel. Timings are given in Table 1 for 
various size images and various numbers of 
labels. 

4. RESULTS 

Figure 1 shows the results of this algorithm 
on a test image. In this example, we assume that 
the "ideal" image (or "ground truth") is 
composed of two distinct regions and we choose 
some compatibility coefficients which reflect this 
assumption; the initial probabilities present three 
classes, two classes corresponding to the two 
"ideal" regions and one "artefact" class. The edge 
image presents strong edges at the border 
between the two "ideal" regions and the edge 
magnitudes decrease with the distance to this 
border. For this example, we notice that if the 
relaxation algorithm is utilized without any edge 
information, one of the labels "takes over" the 
whole image after only three iterations. If the 
edge information is integrated in the relaxation 
process, the two regions are still separated after 
10 iterations and the labeling seems to be stable. 

Figures 2 to 5 present results of the 
algorithm applied on a Landsat-TM scene 
("Washington D.C. region") shown at the top left 
comer of Figure 2. Initial probabilities for this 
scene have been obtained from a classification 
into seven labels, utilizing a probabilistic neural 
network (see (Chettri, 1993) for details). The 7 
labels correspond to the classes "urban", 
"agriculture", "rangeland", "forest", 
"waterbodies", "wetland", and "bareland". 


The algorithm described in section 2 was 
applied to this initial classification, using two 
different numbers of labels; first, we grouped 
these 7 labels into the 3 classes "urban", 
"agriculture", and "other". Figures 2 and 3 show 
the results without and with the edge 
information. When no edge information is 
utilized (Figure 2), one of the labels (label 0) has 
"disappeared" after 20 iterations. When the edge 
information is taken into account (Figure 3), the 
three labels are still present after 30 iterations, 
and the labeling seems to stay stable after 20 
iterations. 

Then, the 7 labels defined previously are 
considered, and we obtain similar results: see 
Figures 4 and 5. When no edge information is 
utilized, only three labels are left after 30 
iterations and one of the three is covering most of 
the image; but when the edge information is 
integrated in the updating formula, all the initial 
labels are still represented after 30 iterations and 
the labeling seems to stabilize after 20 iterations. 
Also, the results obtained after 10 or 30 iterations 
can be compared to the ground truth data shown 
at the top left comer of Figure 5: qualitatively, we 
can observe an overall improvement of the 
segmentation (e.g., suppression of small isolated 
pixels or groups of pixels), but some features 
(e.g., a road) have been regrouped with a 
neighboring region. A quantitative evaluation of 
these results will be performed later. 

The previous results show the importance 
of the edge information and how the integration 
of edge- and the pixel-based segmentations can 
improve the final result. 

Other similar results have also been 
obtained on AVHRR data and will be presented 
at the conference. 
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5. CONCLUSION 

The results presented in this paper show 
how the integration of complementary 
information, such as pixel- and edge-based 
techniques, can improve the final segmentation. 

More work needs to be done, especially in 
the definition of the Initial probabilities, in the 
choice of the compatibility coefficients, and in the 
quantitative evaluation of the results. Also, the 
results presented in section 4 seem to show that, 
when the edge information is utilized, the 
relaxation process becomes "stable" after a 
certain number of iterations: this issue of 
"convergence" will be studied, and different 
schemes, such as the one presented in (Faugeras, 
1981) will be investigated. 
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Image Size 


# Labels 


Edges ? 


Time per 
Iteration 


256*256 7 no 0.36 

256*256 7 yes 0.41 

256*256 3 no 0.10 

256*256 3 yes 0.11 

512*512 2 no 0.17 

512*512 2 yes 0.17 

Tafrk.1 

Timings Obtained for One Relaxation Iteration on a MasPar MP-1 
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"Ideal" 


Edges 


Initial Probabilities 



Iter 1/ no edges Iter 21 no edges Iter 31 no edges Iter 10/ no edges 



Iter 1/ edges Iter 2/ edges Iter 3lf edges Iter 10/ edges 


Figure 1 

Results of our Method on a Test Example 
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Figure 1 

Relaxation Results Without Edge Information 
on a Landsat-TM scene (3 Labels) 
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Figure i 

Results of the " Relaxation With Edges" Method 
on a Landsat-TM scene (3 Labels) 


142 




Figure 4 

Relaxation Results Without Edge Information 
on a Landsat-TM scene (7 Labels ) 
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